fix(server): paginate large thread history to stop the server running out of memory - #3510
fix(server): paginate large thread history to stop the server running out of memory#3510olafura wants to merge 13 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Repository UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Address review (PR pingdotgg#3510, Cursor Bugbot): - Reconnect / checkpoint revert can re-snapshot or filter the live activity window, but the prepended `olderActivities` weren't invalidated — leaving gaps or showing reverted history. Reset the lazy-load state when the live window's oldest activity id changes (it's stable while activities only append, so this doesn't fire during a normal turn), in addition to on thread switch. Web + mobile. - Web only deduped a new older page against already-loaded older pages, not the live window (mobile already did both). Dedup against both so a boundary overlap can't produce duplicate ids / React keys. Not changed — the "unsequenced cursor hides sequenced history" finding is a false positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor ordering, so the oldest-loaded row is only unsequenced once every sequenced row is already loaded; the unsequenced cursor can never strand sequenced rows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Thanks Bugbot — addressed in cddc7e1:
|
ApprovabilityVerdict: Needs human review Introduces a new RPC endpoint and pagination infrastructure across server and clients, fundamentally changing how thread activities are loaded (windowed to 500 with lazy-load). While addressing memory concerns, this is substantial new capability with cross-cutting runtime behavior changes rather than a simple bug fix. You can customize Macroscope's approvability policy. Learn more. |
Address review (PR pingdotgg#3510, Cursor Bugbot): - Reconnect / checkpoint revert can re-snapshot or filter the live activity window, but the prepended `olderActivities` weren't invalidated — leaving gaps or showing reverted history. Reset the lazy-load state when the live window's oldest activity id changes (it's stable while activities only append, so this doesn't fire during a normal turn), in addition to on thread switch. Web + mobile. - Web only deduped a new older page against already-loaded older pages, not the live window (mobile already did both). Dedup against both so a boundary overlap can't produce duplicate ids / React keys. Not changed — the "unsequenced cursor hides sequenced history" finding is a false positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor ordering, so the oldest-loaded row is only unsequenced once every sequenced row is already loaded; the unsequenced cursor can never strand sequenced rows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cddc7e1 to
75d25e2
Compare
|
Addressed the latest review in
Server/web/mobile typecheck, lint, and the projection + timeline suites pass. |
Address review (PR pingdotgg#3510, Cursor Bugbot): - Reconnect / checkpoint revert can re-snapshot or filter the live activity window, but the prepended `olderActivities` weren't invalidated — leaving gaps or showing reverted history. Reset the lazy-load state when the live window's oldest activity id changes (it's stable while activities only append, so this doesn't fire during a normal turn), in addition to on thread switch. Web + mobile. - Web only deduped a new older page against already-loaded older pages, not the live window (mobile already did both). Dedup against both so a boundary overlap can't produce duplicate ids / React keys. Not changed — the "unsequenced cursor hides sequenced history" finding is a false positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor ordering, so the oldest-loaded row is only unsequenced once every sequenced row is already loaded; the unsequenced cursor can never strand sequenced rows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
75d25e2 to
618df70
Compare
|
Addressed the latest review round: Cursor — "Append clears lazy-loaded history" (Medium) — The reset effect used macroscope — mixed-thread flash (Low, The two earlier macroscope Mediums (
|
618df70 to
c8e6474
Compare
|
Latest round: Cursor — "Wrong pagination cursor after live sort" (High). Valid consistency gap I introduced last round. I'd made the reshape sentinel order-independent ( Cursor — "Stale hasMore after revert" (Medium). False positive.
|
c8e6474 to
848cff6
Compare
Self-review (10-angle, max effort) of
|
| Sev | Finding | Fix |
|---|---|---|
| Low (hardening) | The internal SQL-input schemas typed beforeSequence/limit as bare Schema.Number, looser than the contract's NonNegativeInt. The WHERE clause (sequence < beforeSequence OR sequence IS NULL) is only equivalent to the old COALESCE(sequence,-1) < beforeSequence for non-negative cursors — a negative one silently returns only unsequenced rows. |
Tightened both internal schemas to NonNegativeInt so any future non-RPC caller is validated, not just the RPC boundary. Typecheck + 11 pager tests green. |
Latent / by-design (NOT changed — flagging for a decision)
- Window vs reducer NULL-ordering disagreement (highest-severity-if-triggered): the server window orders
sequence DESC(SQLite → NULLs last/oldest) while the client reducer'sactivityOrderusessequence ?? MAX_SAFE_INTEGER(NULLs newest). They only agree today because every row is NULL. Ifsequenceis ever partially backfilled, a thread with mixed rows would (a) drop the newest unsequenced rows from the 500-window and (b) paginate in a different order than it displays. The robust fix is to order both sides bycreated_atconsistently — but that's an index-strategy change (the covering index leads withsequence) and a no-op today, so I've left it. Recommend either ordering bycreated_atend-to-end, or guaranteeingsequenceis all-or-nothing. - Speculative sequence machinery: given the above, the entire sequenced-cursor arm (second SQL query, union input arm,
"beforeSequence" in inputbranch, the sequence-leading index) is dead in practice. A single(created_at, activity_id)keyset would cover 100% of today's reality with ~half the surface. Kept as forward-looking, but worth a conscious call. - web/mobile duplication (AGENTS.md "Duplicate logic across multiple files … should be avoided"): the ~120-line lazy-load state machine (gen guard, in-flight ref, reshape
useLayoutEffect, dedup) is near-verbatim inChatView.tsxanduse-thread-composer-state.ts. Should become one shared hook — deferred because it's a cross-app React-architecture refactor, not a surgical fix. - Asymmetry: activities are windowed to 500 but
messagesin the samegetThreadDetailByIdare still unbounded, so the heap-OOM rationale is only half-applied. handleScroll/loadOlderidentity churn on every live append (perf, not correctness);THREAD_DETAIL_ACTIVITY_WINDOWoverloaded as window size + default page + max page.
Verified safe (no action)
getSnapshot→getCommandReadModel (only the CLI reads .projects); toReversed() (Node/lib supports it); mapThreadActivityRow typing; hasMoreActivities optional doesn't break fixtures/decoders; window slice/hasMore arithmetic; the empty-page stop guard; sequence===0 cursor; created_at tie-break cursor. No CLAUDE.md/AGENTS.md violations beyond the duplication note.
848cff6 to
ead248b
Compare
Addressed the latest review round (
|
Address review (PR pingdotgg#3510, Cursor Bugbot): - Reconnect / checkpoint revert can re-snapshot or filter the live activity window, but the prepended `olderActivities` weren't invalidated — leaving gaps or showing reverted history. Reset the lazy-load state when the live window's oldest activity id changes (it's stable while activities only append, so this doesn't fire during a normal turn), in addition to on thread switch. Web + mobile. - Web only deduped a new older page against already-loaded older pages, not the live window (mobile already did both). Dedup against both so a boundary overlap can't produce duplicate ids / React keys. Not changed — the "unsequenced cursor hides sequenced history" finding is a false positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor ordering, so the oldest-loaded row is only unsequenced once every sequenced row is already loaded; the unsequenced cursor can never strand sequenced rows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ead248b to
311b469
Compare
Address review (PR pingdotgg#3510, Cursor Bugbot): - Reconnect / checkpoint revert can re-snapshot or filter the live activity window, but the prepended `olderActivities` weren't invalidated — leaving gaps or showing reverted history. Reset the lazy-load state when the live window's oldest activity id changes (it's stable while activities only append, so this doesn't fire during a normal turn), in addition to on thread switch. Web + mobile. - Web only deduped a new older page against already-loaded older pages, not the live window (mobile already did both). Dedup against both so a boundary overlap can't produce duplicate ids / React keys. Not changed — the "unsequenced cursor hides sequenced history" finding is a false positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor ordering, so the oldest-loaded row is only unsequenced once every sequenced row is already loaded; the unsequenced cursor can never strand sequenced rows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…#3510) Source: pingdotgg#3510 Source SHA: 034f493 Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks. Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes. Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
…#3510) Source: pingdotgg#3510 Source SHA: 034f493 Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks. Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes. Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
…#3510) (#35) Source: pingdotgg#3510 Source SHA: 034f493 Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks. Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes. Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
Address review (PR pingdotgg#3510, Cursor Bugbot): - Reconnect / checkpoint revert can re-snapshot or filter the live activity window, but the prepended `olderActivities` weren't invalidated — leaving gaps or showing reverted history. Reset the lazy-load state when the live window's oldest activity id changes (it's stable while activities only append, so this doesn't fire during a normal turn), in addition to on thread switch. Web + mobile. - Web only deduped a new older page against already-loaded older pages, not the live window (mobile already did both). Dedup against both so a boundary overlap can't produce duplicate ids / React keys. Not changed — the "unsequenced cursor hides sequenced history" finding is a false positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor ordering, so the oldest-loaded row is only unsequenced once every sequenced row is already loaded; the unsequenced cursor can never strand sequenced rows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…fixes - OSC 10/11/12 rgb in output is the legitimate set-colour command — kept (stripFromOutput flip in the sequence grammar), reply form still stripped from input. - sanitizePersistedTerminalHistory bypasses overflow recovery: whole-buffer loads preserve an over-cap unterminated tail byte-for-byte. - CPR input strip is query-gated (lastCursorQueryRelayedAt + 5s grace): the byte-identical modified-F3 keystroke passes at a quiet prompt while the querying-prompt flood stays broken. - Mobile surfaces the NEWEST open approval/user-input request instead of the oldest, so lazy-loaded ancient requests can't hijack the current prompt. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address review (PR pingdotgg#3510, Cursor Bugbot): - Reconnect / checkpoint revert can re-snapshot or filter the live activity window, but the prepended `olderActivities` weren't invalidated — leaving gaps or showing reverted history. Reset the lazy-load state when the live window's oldest activity id changes (it's stable while activities only append, so this doesn't fire during a normal turn), in addition to on thread switch. Web + mobile. - Web only deduped a new older page against already-loaded older pages, not the live window (mobile already did both). Dedup against both so a boundary overlap can't produce duplicate ids / React keys. Not changed — the "unsequenced cursor hides sequenced history" finding is a false positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor ordering, so the oldest-loaded row is only unsequenced once every sequenced row is already loaded; the unsequenced cursor can never strand sequenced rows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
034f493 to
8494b3e
Compare
…#3510) (#35) Source: pingdotgg#3510 Source SHA: 034f493 Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks. Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes. Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
…#3510) (#35) Source: pingdotgg#3510 Source SHA: 034f493 Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks. Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes. Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
…#3510) (#35) Source: pingdotgg#3510 Source SHA: 034f493 Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks. Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes. Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
Address review (PR pingdotgg#3510, Cursor Bugbot): - Reconnect / checkpoint revert can re-snapshot or filter the live activity window, but the prepended `olderActivities` weren't invalidated — leaving gaps or showing reverted history. Reset the lazy-load state when the live window's oldest activity id changes (it's stable while activities only append, so this doesn't fire during a normal turn), in addition to on thread switch. Web + mobile. - Web only deduped a new older page against already-loaded older pages, not the live window (mobile already did both). Dedup against both so a boundary overlap can't produce duplicate ids / React keys. Not changed — the "unsequenced cursor hides sequenced history" finding is a false positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor ordering, so the oldest-loaded row is only unsequenced once every sequenced row is already loaded; the unsequenced cursor can never strand sequenced rows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…fixes - OSC 10/11/12 rgb in output is the legitimate set-colour command — kept (stripFromOutput flip in the sequence grammar), reply form still stripped from input. - sanitizePersistedTerminalHistory bypasses overflow recovery: whole-buffer loads preserve an over-cap unterminated tail byte-for-byte. - CPR input strip is query-gated (lastCursorQueryRelayedAt + 5s grace): the byte-identical modified-F3 keystroke passes at a quiet prompt while the querying-prompt flood stays broken. - Mobile surfaces the NEWEST open approval/user-input request instead of the oldest, so lazy-loaded ancient requests can't hijack the current prompt. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…#3510) (#35) Source: pingdotgg#3510 Source SHA: 034f493 Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks. Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes. Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
…#3510) (#35) Source: pingdotgg#3510 Source SHA: 034f493 Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks. Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes. Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
8494b3e to
d7ddba7
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit d7ddba7. Configure here.
Address review (PR pingdotgg#3510, Cursor Bugbot): - Reconnect / checkpoint revert can re-snapshot or filter the live activity window, but the prepended `olderActivities` weren't invalidated — leaving gaps or showing reverted history. Reset the lazy-load state when the live window's oldest activity id changes (it's stable while activities only append, so this doesn't fire during a normal turn), in addition to on thread switch. Web + mobile. - Web only deduped a new older page against already-loaded older pages, not the live window (mobile already did both). Dedup against both so a boundary overlap can't produce duplicate ids / React keys. Not changed — the "unsequenced cursor hides sequenced history" finding is a false positive: NULL-sequence (legacy) rows always sort oldest in the window/cursor ordering, so the oldest-loaded row is only unsequenced once every sequenced row is already loaded; the unsequenced cursor can never strand sequenced rows. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…fixes - OSC 10/11/12 rgb in output is the legitimate set-colour command — kept (stripFromOutput flip in the sequence grammar), reply form still stripped from input. - sanitizePersistedTerminalHistory bypasses overflow recovery: whole-buffer loads preserve an over-cap unterminated tail byte-for-byte. - CPR input strip is query-gated (lastCursorQueryRelayedAt + 5s grace): the byte-identical modified-F3 keystroke passes at a quiet prompt while the querying-prompt flood stays broken. - Mobile surfaces the NEWEST open approval/user-input request instead of the oldest, so lazy-loaded ancient requests can't hijack the current prompt. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…#3510) (#35) Source: pingdotgg#3510 Source SHA: 034f493 Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks. Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes. Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
…#3510) (#35) Source: pingdotgg#3510 Source SHA: 034f493 Imported: native mobile lazy loading for older thread activity, a 1,000-event subscription catch-up ceiling with snapshot fallback, and synchronized stale snapshot watermarks. Adapted: applied above the refreshed pingdotgg#4018 web/server candidate and preserved Tim lifecycle handling plus our mobile composer changes. Excluded: pingdotgg#3510 server/web pagination duplicated by pingdotgg#4018, the later shared-hook refactor, formatting-only commits, and contract comments. The shared refactor can be revisited independently after production validation.
|
Closing in favor of #2829 (orchestration V2). #2829 deletes the V1 orchestration layer this PR builds on — This is not a judgement on the change itself. Several of these are real gaps we still want fixed; the base just moved out from under them. Once #2829 merges, please rebase onto |

Fixes the server-side root cause behind #2761 (and likely #996).
What Changed
The server used to load a thread's entire history — every tool activity, message, and checkpoint — into memory at once. On a busy database that's hundreds of MB, which exhausted the Node heap and crashed the server (and made large threads slow to load / churn reconnects, as reported in #2761).
Now the server loads only the most recent slice of a thread's activity and tells the client whether older history exists; the web and mobile apps fetch older pages on demand as you scroll up. Deep history is still fully reachable — it's paged in instead of loaded all at once.
t3CLI offline path stop loading every thread's full history (they only read the project list). Thread detail is bounded to the most recent activities, with ahasMoreActivitiesflag and a cursor-paginatedgetThreadActivitiesRPC for older pages. The older-page query is shaped to use the existing index instead of a full sort.The commits remain separated by server, web, mobile, shared-client, reconnect-safety, and rebase-formatting concerns so each layer is independently reviewable.
Why
Programs in a thread emit a lot of tool activity over time. The server treated "open a thread" (and "load the snapshot") as "read all of it from SQLite into memory." For a long-lived thread or a busy database, that single read materialised more than the heap could hold, so the process ran out of memory and died — and even when it didn't crash, shipping one giant snapshot blocked the connection (#2761). Bounding the read + paging older history on demand keeps memory flat regardless of how much history a thread accumulates.
UI Changes
The web and mobile timelines gain a small "Load older history" affordance and a loading spinner when you scroll to the top of a thread longer than the window. It's a minor, additive interaction. I wasn't able to attach before/after screenshots or a scroll video here — happy to add them, or the server commit alone (no UI) carries the actual reliability fix if you'd rather review that first.
Checklist
🤖 Generated with Claude Code
Note
High Risk
Changes core read paths (bounded thread detail, snapshot/CLI switching to command read model, subscribe catch-up behavior) and pagination edge cases for sequenced vs legacy activities; any consumer expecting a full snapshot or unbounded replay could break.
Overview
Server memory and API: Thread detail now loads only the 500 most recent activities and sets
hasMoreActivitieswhen more exist.getThreadActivitiespages older rows via sequence or legacy(createdAt, activityId)cursors. The orchestration HTTP snapshot andt3CLI offline path usegetCommandReadModelinstead of materializing full per-thread tables.subscribeThreadreplays at most 1,000 global events after the cursor; otherwise it emits a replacement thread snapshot (with live events buffered so nothing is lost).Clients: New
useOlderThreadActivitiesinclient-runtimeprepends paged history, dedupes, and resets on live-window reshape. Web and mobile scroll-up (and explicit controls) load older pages; the feed and composer use merged activities so approvals/user-input prompts still surface after scrolling back. Mobile picks the newest open pending request when multiple exist in the merged set. Sidebar thread prewarming is removed.Contracts:
OrchestrationThreadgains optionalhasMoreActivities; HTTP snapshot success type documents empty thread payloads on the command read model.Reviewed by Cursor Bugbot for commit eac66d8. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Paginate large thread activity history to prevent server memory exhaustion
hasMoreActivitiesflag onOrchestrationThreadpayloads when older history exists.orchestration.getThreadActivitiesWebSocket RPC to fetch older activities using a sequence-based or legacy unsequenced (createdAt/id) cursor, with a max page size of 500.subscribeThreadcatch-up replay to 1000 events; for stale or invalid cursors, sends a thread snapshot followed by buffered live events instead of attempting unbounded replay.useOlderThreadActivitieshook inclient-runtimeand wires it intoChatView(web) anduseThreadComposerState(mobile) to support infinite scroll-up with deduplication and reshape detection.MessagesTimeline(web) andThreadFeed(mobile) components now render a 'Load older history' button or loading indicator at the top, with auto-load on scroll to start.orchestration/snapshotHTTP endpoint and offline CLI snapshot now return the command read model (getCommandReadModel) instead of the full snapshot, which is a breaking change for any consumers relying on the full activity history from that endpoint.Macroscope summarized eac66d8.